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Abstract 

' Particle swarm optimization (PSO) is a widely used nature-inspired meta-heuristic for solving 

continuous optimization problems. However, when running the PSO algorithm, one encounters 
' the phenomenon of so-called stagnation, that means in our context, the whole swarm starts to 

converge to a solution that is not (even a local) optimum. The goal of this work is to point 
out possible reasons why the swarm stagnates at these non-optimal points. To achieve our 
results, we use the newly defined potential of a swarm. The total potential has a portion for 
every dimension of the search space, and it drops when the swarm approaches the point of 
convergence. As it turns out experimentally, the swarm is very likely to come sometimes into 
"unbalanced" states, i. e., almost all potential belongs to one axis. Therefore, the swarm becomes 
blind for improvements still possible in any other direction. Finally, we show how in the light 
of the potential and these observations, a slightly adapted PSO rebalances the potential and 
therefore increases the quality of the solution. 

. 

O ■ 1 Introduction 

m 

Background. Particle swarm optimization (PSO), introduced by Kennedy and Eberhart [KE95, 
EK95], is a very popular meta-heuristic for solving continuous optimization problems. It is inspired 
^ ; by the social interaction of individuals living together in groups and supporting and cooperating 

with each other. Fields of very successful application are, among many others, in Biomedical Image 
Processing }WSZ + 04| . Geosciences |OD10j . Mechanical Engineering [GWHK09] . and Materials Sci- 
ence [RPPN09], to name just a few, where the continuous objective function on a multi-dimensional 
domain is not given in a closed form, but by a "black box." The popularity of the PSO framework 
in these scientific communities is due to the fact that it on the one hand can be realized and, 
if necessary, adapted to further needs easily, but on the other hand shows in experiments good 
performance results with respect to the quality of the obtained solution and the speed needed to 
obtain it. By adapting its parameters, users may in real- world applications easily and successfully 
control the swarm's behavior with respect to "exploration" ("searching where no one has searched 
before") and "exploitation" ("searching around a good position"). A thorough discussion of PSO 
can be found in [PSLllj . 

To be precise, let an objective function / : M. D — > M. on a D-dimensional domain be given 
that (w. 1. o. g.) has to be minimized. A population of particles, each consisting of a position (the 



1 



candidate for a solution), a velocity and a local attractor, moves through the search space H D . The 
local attractor of a particle is the best position with respect to / this particle has encountered 
so far. The population in motion is the swarm. In contrast to other evolutionary algorithms, 
the individuals of the swarm cooperate by sharing information about the search space via the 
global attractor, which is the best position any particle has found so far. The particles move in 
time-discrete iterations. The movement of a particle is governed by both its velocity and the two 
attractors and by some additional fixed parameters (for details, see Sec. [2j). 

PSO is widely used in real- world applications. It is usually desired that the swarm converges to a 
single point in the search space, and that this point is at least a local optimum. It is well investigated 
how the fixed parameters mentioned above should be chosen to let the swarm converge at all |Tre031 
However, experiments sometimes show the phenomenon of stagnation, meaning here that 
the particle swarm sometimes converges to a single search point and gets stuck, although this point 
is not even a local optimum. In [LWllj . Lehre/Witt have for a certain situation formally proven that 
the probability for a swarm to converge to a non-optimal point is positive. Several approaches to 
deal with this stagnation phenomenon have been developed. Clerc [DIeQ6] examines the distribution 
of the particles during the stagnation phase, derives properties of these distributions, and provides 
several possibilities to adapt the algorithm for the case that the number of iterations without 
an improvement reaches a certain threshold. Van den Bergh/Engelbrecht [vdBE02 substantially 
modify the movement equations, enabling the particles to count the number of times they improve 
the global attractor and use this information. Empirical evidence for the capability of their method 
to find local optima on common benchmarks is given. Closest to our work, in [LWllj the movement 
equations are modified by adding in every iteration a small random perturbation to the velocity. 

New results. In this paper, we focus on the swarm's behavior right before the convergence starts, in 
order to find out about possible causes that let the swarm converge, i.e., why the global attractor 
starts to stagnate. Although one would like the output of the PSO algorithm to be at least a 
local optimum, we point out two possible reasons for a swarm to converge far away from any 
local optimum. In order to state the causes for this premature convergence, we define a potential 
that reflects the capability of the swarm to move. That means the swarm converges iff the total 
potential approaches 0. The swarm's total potential has a portion for every dimension of the 
L>-dimensional search space. The experiments carried out suggest that unwanted stagnation can 
indeed be explained in terms of the potential. 

The first possible cause of a non-optimal limit of convergence is that, even though the global 
attractor is updated frequently, the potential drops, so the swarm has not sufficient momentum to 
find significantly improved points. In Sec. El we present experiments that demonstrate that this 
phenomenon can in fact be observed for some parameter selections. Fortunately, it also turns out 
that common parameter choices and a reasonable swarm size already avoid this problem. 

The second and more important reason is that the potential tends to becoming imbalanced 
among the dimensions, so dimensions in which only small improvement is possible may neverheless 
have the by far highest potential. That means that every step in such a dimension results in a 
worsening strong enough to void possible improvements in other dimensions. So, the global attrac- 
tor of the swarm stagnates and the swarm starts to converge. We demonstrate that indeed the 
swarm tends to reach a state where the potentials are unbalanced, i. e., one dimension gets the by 
far highest portion of the total potential while all other portions are about equal. Then, we present 

In a companion paper to the paper at hand, the quality of the best point found by the swarm (the global attractor) 
is formally analyzed. 
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experimental evidence showing that this phenomenon makes the particles converge at non-optimal 
search points. So the experiments suggest that first the swarm starts to prefer a promising direction 
that is parallel to one of the axes and increases the potential in this dimension far above the poten- 
tials in the other dimensions. As soon as the chosen direction does no longer yield improvements, 
its potential stays much larger than in the remaining dimensions where improvements would still 
be possible. From that point on, improvements become rare and the swarm starts to stagnate, 
although no local optimum is found yet. 

Since the cause of this premature convergence is an imbalance of the potentials, we show how 
a small, simple and easy to implement modification of the algorithm enables it to handle such 
situation. Namely, if the potential is sufficiently small, we let the particles make pure random steps, 
which do not favor any direction. We conclude with showing that the modification does not totally 
overwrite the PSO algorithm by replacing it by some greedy random search procedure. Instead, our 
experiments show that the modification is only applied in case of (premature) convergence. As long 
as there is still room for improvements left in the search space, the unmodified behavior prevails. 

2 Definitions 

First we present the underlying model of the PSO algorithm. The model describes the positions of 
the particles, the velocities and the global and local attractors as real-valued stochastic processes. 
Furthermore, we define in Def. [2] the potential of a swarm that will be a measure for its movement. 
A swarm with high potential is more likely to reach search points far away from the current global 
attractor, while a swarm with potential approaching is converging. 

Definition 1 (Classical PSO Algorithm). A swarm S of N particles moves through the D- dimensional 
search space M. D . Each particle n consists of the following components: 

• position X n 6 M> D , describing the current location of the particle in the search space, 

• velocity V n £ ~M, D , describing the vector of the particle's current velocity, 

• local attractor L n 6 MP , describing the best point particle n has visited so far. 

Additionally, the swarm shares information via the global attractor G E MP , describing the best 
point any particle has visited so far. After some arbitrary initialization of the X n and V n (usually, 
one assumes them to be initialized u. a. r. over some domain), the actual movement of the swarm 
is governed by the procedure described in Algorithm [7] where f : M D — > K denotes the objective 
function. 

Here, \> c i an d c 2 are some positive constants called the fixed parameters of the swarm, and 
randQ returns values that are uniformly distributed over [0, 1] and all independent. 

Note that in case of a tie between the previous attractor and the new point X n , we use the new 
value, i. e., whenever a particle finds a search point with value equal to the one of its local attractor, 
this point becomes the new local attractor. If additionally the function value is equal to the one 
of the global attractor, this one is also updated. Also note that the global attractor is updated as 
soon as a new better solution has been found. 

Now we want to define a potential for measuring how close the swarm is to convergence. A 
meaningful potential should of course involve the velocities of the particles. These considerations 
lead to the following definition: 
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Algorithm 1: classical PSO 



output : G G 
1 repeat 

for n = 1 — > N do 
for d = 1 — > D do 

V?:=X-ty + ciTand().(L3 
+ c 2 • randQ • (G d 

end 

if f{X n ) < f(L n ) then 
I L n := X n ; 
end 

if f{X n ) < /(G) then 
I G:=X n ; 
end 
end 

14 until termination criterion met; 



Definition 2 (Potential). For d E {1, . . . , -D}, the potential of swarm S in dimension d is &d with 

N 

n=l 

the total potential of S is $ = ($X; ■ ■ ■ ?$d)- 

So the current total potential of a swarm has a portion in every dimension. Between two 
different dimensions, the potenial may differ much, and "moving" potential from one dimension to 
another is not possible. On the other hand, along the same dimension the particles influence each 
other and can transfer potential from one to the other. This is the reason why we do not define a 
potential of an individual particle. 



3 1-Dimensional PSO 

In this section, we examine the behavior of a 1-dimensional PSO with respect to the potential. 
If the swarm is close to a local optimum and there is no second local optimum within range, 
the attractors converge and it is well-known that with appropriate choices for the parameters of 
the PSO, convergence of the attractors implies convergence of the whole swarm. Such parameter 
selection guidelines can be found, e.g., in | JLY07] . 

On the other hand, if the swarm is far away from the next local optimum and the function is 
monotone on an area that is large compared to the current potential of the swarm, the preferred 
behavior of the swarm is to increase the potential and move in the direction that yields the improve- 
ment until a local optimum is surpassed and the monotonicity of the function changes. In [LW11| . 
the authors show that there are non-trivial choices of parameters for which the swarm converges 
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even on a monotone function. In particular, if N = 1, every parameter choice either allows con- 
vergence to an arbitrary point in the search space, or it generally prevents the one-particle-swarm 
from converging, even if the global attractor is already at the global optimum. 

We ran the particle swarm algorithm on a monotone function to measure the development of 
the potential over time. For our experiment, we chose the 1-dimensional function f{x) = —x as 
objective function wanting the swarm always "running down the hill." Note that this choice is not a 
restriction, since the particles compare points only qualitatively and the behavior is exactly the same 
on any monotone decreasing function: due to the rules for updating the attractors in lines 7 and 
11, resp., of Algorithm [H the new attractors are the points with greater x-coordinate. Therefore, 
we used only one function in our experiment. The parameters for the movement equations are 
common choices obtained from the literature. We let the particles make 1000 iterations and stored 
the potential at every iteration. We made a total of 1000 experiments for each set of parameters and 
calculated both average and standard deviation. The averages are stated in Fig. [H the standard 
deviations are of the same order and therefore omitted. In cases (a), (c), (d) and (e), the particles 
have shown the expected behavior, namely an exponential increase of the potential. So the swarm 
keeps running down the hill which is what we want it to do. 




200 400 600 800 1000 1200 

Iteration 

Figure 1: 

(a) x = 0.729, ci = c 2 = 1.49, N = 2 |CK02j 

(b) X = 0.729, ci = 2.8 • X , c 2 = 1.3 • x, N = 2 [CDOl] 

(c) X = 0.729, ci = 2.8 • X, c 2 = 1. 3 ■ x , N = 3 [CDOl] 

(d) x = 0.6, ci = c 2 = 1.7, N = 2 |Tre03| 

(e) x = 0.6, ci = c 2 = 1.7, N = 3 [Tre03j 



However, in case (b) where only two particles are involved, we see the potential decreasing 
exponentially because the number of particles is presumably too small. In this case, the swarm will 
eventually stop, i.e., stagnate. But we also see in case (c) that using one additional particle and 
not changing the remaining parameters, the swarms keeps its motion. 

In all cases, for the small swarm size of > 3, the common parameter choices avoid the problem 
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mentioned in [LW11]. 



4 D-Dimensional PSO 

In the D-dimensional case, the situation is more complicated as now the relations between distinct 
dimensions become important. A new problem arising is the following: Assume that the whole 
swarm is close to a point x € M. D such that every change of the first coordinate leads to a signif- 
icantly worse value of the objective function, while in the other dimensions there is still room for 
improvements. Furthermore let the swarm have high potential in the first and low potential in 
any other dimension. Then an improvement of the global attractor is still possible, but it is very 
unlikely and between two updates are many steps without an update. The reason is that any im- 
provement in some of the dimensions 2, D is voided by the much larger worsening in dimension 
1. It follows that the attractors stay constant for long times between two updates and so the swarm 
tends to converge and therefore looses potential. As long as the global attractor stays constant, the 
situation is symmetric in every dimension, so while converging, the imbalance is still maintained. 

First, we want to examine if and how such imbalances arise. Assume that the fitness function is 
(on some area) monotone in every dimension. One of our main observations is that indeed in such a 
situation the swarm tends to pick one dimension and favor it over all the others. As a consequence, 
the movement of the swarm becomes more and more parallel to one of the axes. 

We used the fitness-functions f(x) = — YliLx x i an< ^ g(%) = — 1la=i * ' x i which ar e both 
monotonically decreasing in every dimension and set D to 10. Initially, we distribute the particles 
randomly over [-100; 100] D and the velocities over [-50;50] D and let the swarm make 500 iterations. 
The swarm size N was 10 and the parameters were set to x = 0.729, c\ = C2 = 1.49 as found 
in [CK02j. After each iteration, we calculated the potential for each dimension. We made 1000 
runs and for each run, the dimensions were sorted according to the final value of i. e., we switched 
the numbers of the dimensions such that after the last iteration dimension 1 always had the highest 
potential, dimension 2 the second highest and so on. We calculated the mean of the potentials over 
the 1000 runs for each of the sorted dimensions. The results are stated in Fig. [2j One can see that 
the dimension with the greatest potential has for both functions a value far higher than the others, 
while the other dimensions do not show such a significant difference between each other. In other 
words: Particles like to move parallel to an axis. 

An explanation for this behavior is the following: Assume that at some time, one dimension do 
has more potential than the others. Further assume that the advance is great enough such that for 
some number of steps the particle with the largest value in dimension d$ is the one that determines 
the global attractor. In a companion paper, we call a swarm in this situation "running". Since 
randomness is involved and this situation has a positive probability to occur, it will actually occur 
after sufficiently many iterations. Then, each update of the global attractor increases the potential 
in do considerably because it increases the distance of every single particle to the global attractor 
except for the one particle that updated it. In any other dimension d ^ do, the situation is different. 
Here, the decision which particle updates the global attractor is stochastically independent of the 
value x<i in dimension d. In other words: If one looks only on the dimension d, the global attractor 
is chosen uniformly at random from the set of all particles. As a consequence, after some iterations, 
the do'th coordinate of the velocity becomes positive for every particle, so the attraction towards 
the global attractor always goes into the same direction as the velocity, while in the remaining 
dimensions, the velocities may as well point away from the global attractor, meaning that the 
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Iteration Iteration 

(a) Fitness function / (b) Fitness function g 

Figure 2: Growth of potential when processing (a) f(x) = — Y^iLi x ii 3 ) 9$) = ~~ T^h=i * ' x «- 

particles will be slowed down by the force of attraction. An overview over the situation is given in 
Fig. El 

So, roughly speaking, most of the time the global attractor is somewhere in the middle of the 
different values, giving less potential increase then in dimension do where it has a border position. 
That means that the balanced situation is not stable in a sense that after the imbalance has reached 
a certain critical value, it will grow unbounded. 

If at some point no more improvements can be made in dimension do, the swarm is in the 
situation described above where it starts to converge while the convergence is, other than the 
acceleration phase, balanced. That means after the same time the potential of every dimension is 
decreased by approximately the same factor, so dimension do has still far more potential than any 
other dimension and the swarm stays blind for possible improvements in dimensions other than d$. 

To supplement the results about the behavior of the PSO in that "artificial" setting, we ran it 
on two well-known benchmark functions to show that the problems described above really occurs 
on actual instances. Since the described scenario may happen with positive but, depending on 
the situation, small probability, we choose the number of particles N compared to the number of 
dimensions D small in order to be able to view the phenomenon in a preferably pure condition. 
Table [TJ lists our results on the sphere function with optimal solution z* = (0, — ,0), where we 
distributed the particles randomly over [-100;100] D and the velocities over [-50,50] D , and an- 
other common benchmark, the Rosenbrock function with optimal solution z* = (1,...,1) (found 
in [Ros60| ) , where the initial population was randomly distributed over [—5; 10] D and the initial 
velocity over [—2.5,5]°. The results obtained on this function are stated in Table [2) We repeated 
each experiment 1000 times and calculated the means. Additionally we calculated for each rep- 
etition the dimension with the minimal and the one with the maximal value for the potential f> 
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I do 

Figure 3: Particles running in direction do- In dimension do, the differences between the coordinate 
of the particle and the global attractor is on average higher than in dimension d\. The velocities 
of dimension do point in the direction of the global attractor. 



after the last iteration (see columns and the difference between the global attractor and the 
optimal solution in the dimension with the lowest resp. highest remaining potential. One can see 
that the dimension with the highest value for $ usually is much closer to its optimal value than 
the dimension with the lower value. In particular, in the 2-dimensional case the potential became 
in one dimension preventing the swarm from any movement in this dimension and consequently 
from finding the minimum. 

Table 1: Sphere- function 



D 




4 


60 


150 


N 




2 


10 


20 


^max 




10000 


100000 


100000 


Value 




51.04 


12.18 


11.97 


min. $ 




0* 


5.84 ■ 10" 62 


4.15 ■ 10" 37 


dist. opt. 


1.58 


1.32 


1.30 


max. $ 


$ 


3.75 ■ 10~ s 


7.53 • 10- s 


1.59 ■ 10 _v 


dist. opt. 


1.16 ■ 10~° 


1.91 ■ 10 _a 


2.28 • 10~ a 



Due to double precision. 



We calculated the standard deviation for each unit and obtained values that were of the same 
order but usually higher than the mean values. The reason for this high deviations is that the 
examined phenomenon occurs randomly and therefore one cannot predict the potential level of the 
swarm when it occurs, i. e., the level of imbalance at the point when the swarm starts to converge 
is unpredictable. 

Now that we know that the swarm tends to optimize functions dimensionwise, it is interesting 
to see what happens if we try it on function that is in some area increasing in every dimension but 
still decreasing in some direction. 

Fix some b > 1 and define the D-dimensional function / as follows: 



T,i=x x h 3i,j:xi>b- 



2 max. 



otherwise 
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Table 2: Rosenbrock- function 



D 




4 


60 


150 


N 




2 


10 


20 


^max 




10000 


100000 


100000 


Value 




126.54 


34.57 


28.88 


min. <E> 


4 


0" 


6.27 ■ 10~ 5 


2.32 ■ 10~ 4 


dist. opt. 


1.1075 


0.37 


0.12 


max. <E> 


4 


4.72 ■ 10" s 


0.93 


8.07 


dist. opt. 


2.59 


0.11 


0.06 



Due to double precision. 




Figure 4: Continuous, not differentiable function / 

In Fig. [J]one can see a plot of / for b = 1.1 and D = 2. For y not between x/b and x ■ b, this 
function behaves like the well-known sphere-function, leading the particles close to the origin. For 
x = y, f(x, y) = —2 ■ x 2 and from y = x/b (y = x ■ b) to y = x, the function falls into a valley. One 
can easily verify that this function is, though not differentiable, at least continuous. One would 
want the particles to be able to move through the valley. 

As in our previous experiment, we set D = 10, x = 0.729, c\ = C2 = 1.49, N = 10 and 
initialized the particles uniformly at random over [—100; 100]^* (except for the first particle that 
was initialized at (1, 1) such that the swarm could see the direction where the improvements are 
possible) and the velocities over [-50;50] D . We let the swarm do 1000 runs with 500 iterations each. 
The potential of the dimension with the highest potential after the last iteration was determined 
and the mean and standard deviation of the respective dimensions were calculated over the 1000 
repetitions. This was done for two values for the parameter b of the fitness function, namely 
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b = 1.00001 and b = 1.000000001. As expected, the capability of the particles to accelerate and 
increase their potential heavily depends on the choice of this parameter. As one can see in Fig. [5j 
the smaller the value of b gets, the smaller the gain of potential. An explanation for this behavior 
is that increasing the potential requires the global attractor to be updated in a certain frequency. 
If the parameter b is too small, the area for improvements are also too small and updates of the 
global attractor happen too rarely. 

We repeated the experiment for b = 1.000000001 and a = fc^, using the fitness function 

Yji=i x h ■ Xi > a ■ xi \/ Xi < —a ■ x\ 



1 ' ' max, 



1 U 



X I 
Xl 



1 , otherwise. 



Note that this function with a set to |fj behaves in some sense similar to / . Both functions 
have areas that allow improvements. That areas have the same shape but the direction where 
improvements of / ro t are possible is the positive xi-direction. Fig. [5c] shows how the swarm 
handles this function much better than the function / before. In particular, this shows that PSO 
is not invariant to rotations of the search space. That is possible because of the capability of the 
swarm to favor one direction only if this direction is parallel to one of the axes. 




5 Modified PSO 

In the previous section, we have seen that the particle swarm might get stuck if its potential is too 
high in dimensions that are already optimized and too low in dimensions that could still be improved. 
Then the global attractor stagnates and the swarm starts to converge. Since the convergence 
happens in a symmetric manner along the different dimensions, the imbalance is maintained. A 
small and simple modification of the PSO algorithm avoids that problem by enabling the swarm to 
rebalance the potentials in the different dimensions: 

Definition 3 (Modified PSO). For some arbitrary small but fixed 5 > 0, we define the modified 
PSO via the same equations as the classic PSO in Def. [7J only modifying the velocity update in line 
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[7] of Algorithm^ to 



y d n ■■= < 



(2 • rand() - 1) • 6, if\/d' € {1, D} : \V$\ + \G d , - X$\ < 5, 



k X ■ V£ + Cl • rand() • (L^ - X£) + c 2 • randQ • (G d - X%), otherwise. 



In words: As soon as the sum of the velocity and the distance between the position and the 
global attractor of a particle are below the bound of 5 in every single dimension, the updated velocity 
of this particular particle is drawn u. a. r. from the interval [—5, 5]. Note the similarity between 
this condition and the definition of the potential. Indeed, we could have used the condition <J><2 < 5 
(with some fixed a) instead, but we decided to keep the modification as simple and independent 
from the terms occurring in the analysis as possible. Now the potential can no longer converge to 
while staying unbalanced because if it decreases below a certain bound, we randomly assign a 
value to the velocity which on expectation has an absolute value of 5/2. If a velocity is assigned 
that way, we call the step forced. 

This modified PSO is similar to the Noisy PSO proposed by Lehre and Witt in [LW11] where 
they generally add a random perturbation drawn u. a. r. from [—6/2,5/2] for some small 5 and 
prove that their swarm is able to find a local optimum. However, their analysis is restricted to one 
specific 1-dimensional fitness function. 

The modification does not prevent the PSO from emerging imbalance between the potentials 
of different dimensions. But the imbalanced convergence phenomenon described above is no longer 
possible. When the global attractor of the modified PSO gets stuck and the potential decreases, 
there will be a point when both the velocities and the distances to the global attractor in every di- 
mension get lower than 5. From that moment on, the relationship between the different dimensions 
gets balanced by the forced steps which on expectation give every dimension the same amount of 
potential. So, the potentials of the different dimensions are made equal. 

We repeated the experiment from the previous section in the same setting as before, but using 
the modified PSO. The results can be seen in Tabled It turns out that the modified PSO algorithm 
actually leads to a better solution than the unmodified one. 

Table 3: Comparison between the classic and the modified PSO algorithm 



Function 


D 


N 


imax 


s 


Value 


Sphere 


4 


2 


10000 


io- ia 


43.34 


Sphere 


4 


2 


10000 




51.04 


Sphere 


60 


10 


100000 




4.07 


Sphere 


60 


10 


100000 




12.18 


Sphere 


150 


20 


100000 


w- L ' 2 


6.41 


Sphere 


150 


20 


100000 




11.97 


Rosenbrock 


4 


2 


10000 


10-'' 


8.80 


Rosenbrock 


4 


2 


10000 




126.54 


Rosenbrock 


60 


10 


100000 


io- v 


2.02 


Rosenbrock 


60 


10 


100000 




34.57 


Rosenbrock 


150 


20 


100000 


io- a 


2.25 


Rosenbrock 


150 


20 


100000 




28.88 



1 Due to double precision. 



We also calculated the standard deviation for each unit and obtained values that were of the 
same order but usually higher than the mean values. The reason for this high deviations is that 
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the examined phenomenon occurs randomly and therefore one cannot predict the potential level of 
the swarm when it occurs. 

To make sure that the modification does not fully take over, we plotted the forced points with 

5 = 10~ 7 and the 2-dimensional sphere function as objective function in Fig. [6j As can be seen 
in the figure, the particles get forced near (—2 • 10~ 5 ,0) but their movement does not stay forced. 
Instead, the swarm becomes running again until the particles approached the optimum at (0,0). 
This implies that for sufficiently smooth functions, the modification does not take over, replacing 
the PSO by some random search routine. Instead, the modification just helps to overcome "corners" . 
As soon as there is a direction parallel to an axis with decreasing function value, the swarm becomes 
"running" again and the unmodified movement equations apply. 

6 Conclusion 

This paper focuses on the behavior of a particle swarm to find good regions in the search space. We 
found out that the potentials of the different dimensions are most likely to become unbalanced and 
that this imbalance possibly causes the particle swarm to get stuck at non-optimal search points. 
A suggestion to modify the algorithm by randomly assigning a small velocity if the potential of a 
particle falls below a certain bound is suggested. Additionally, we show that the modification does 
not take over the swarm, it just corrects the direction before the classic movement equations are 
applied again. 
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